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Abstract 

An approach used to determine some university faculty annual raises is described. 
Faculty assessment information is covered first. Tabular summaries made from this information 
are also described. Then the annual evaluation procedure carried out by a Faculty Review Team 
and the Department Head are described. The method used to determine faculty raise allocations 
is presented. Difficulties and limitations of the approach are mentioned throughout. 
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An Approach to Annual Assessment 
and Evaluation of University Faculty 

Introduction 

The evaluation of annual university faculty performance is a difficult and, for some 
individuals, sensitive issue. The natural precursor to faculty evaluation, faculty assessment, may 
also be quite involved for individuals on both “ends.” Most of the professional writings located 
that pertain to faculty assessment and evaluation appear to focus on classroom instruction - this 
conclusion was reached after scanning the reference lists in Braskamp and Ary (1994), Centra 
(1993), Lucas (1994), and recent volumes of the iQ umal of Personnel Evaluation in 
An assortment of ideas on faculty assessment and evaluation in general is given in Diamond and 
Adam (1993), while Creswell (1986) focuses on assessing faculty research work. A survey 
pertaining to what information has been considered in the faculty evaluation process is discussed 
by Seldin (1993). Various methods of determining salary allocations (including point allocation, 
percent designation, regression analysis, plus various other calculation methods) are scholarly 
discussed by Camp, Gibbs, and Masters (1988). Their particular emphasis was on a “finite 
increment” model which includes human capital considerations, base salary adjustments, merit 
points, market-based adjustments, and a rather involved formula for determining allocations. 

The purpose of the current manuscript is to describe an approach to the annual 
assessment and evaluation of faculty members that I have used (and refined in an iterative 
manner) over the past seven years. The context considered herein is the Department of 
Educational Psychology at The University of Georgia. The information collected for each 
faculty member is discussed first, followed by descriptions of summary information and the 
review process, and finalized with a discussion of the allocation of raise monies. Some 
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concluding comments are added at the end. 
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Basic Information Collected 

Calendar year activities of each faculty were put into three file folders. File A contained 
three items: a College of Education form, the assigned work load, and a Self Assessment (also 
called Statement of Productivity and Professional Development). The COE form was recently 
revised to become quite complete. It consists of eight sections: Instructional Activities, Awards, 
Honors and Recognitions, Research and Other Creative Activities, Contracts and Grants, 

Profess ionalAJniversity Service Activities, University/College Initiatives, and Administrative 
Responsibilities. Any activities of a faculty member that could not be reported in the College 
form could be included in the two-to-four-page Self Assessment. 

File B contained information related to instruction. Copies of student feedback 
(numerical and written comments) for sections during the calendar year under consideration 
were supplied by the Department. The Department staff also summarized course sections taught 
by each faculty member: course number and credit hours, responses to six items on the College 
Course Evaluation Form (completed by class students), and grade distribution for each class 
(which indicated the number of students in each class). Each faculty member was given the 
option of turning in any course material, including syllabi, tests, and handouts. Some faculty 
members opted to view this as an opportunity to present their “portfolios” on instruction. [A 
more fomial “professional portfolio” that could reflect many aspects of a faculty member’s 

annual productivity is discussed by Froh, Gray, and Lambert (1993).] 

File C contained each faculty member’s professional writings. This included books, 

chapters, and journal articles that appeared in print during the calendar year under consideration. 
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The file could also contain writings in press or submitted for publication, as well as complete 
papers that were presented at professional meetings. 

The proportions of time assigned to the four categories of instruction, research, service, 
and administration were determined as follows. Under the quarter system (which we had prior 
to 1998) there were 12 “units” for a calendar year, three per term. Under the semester system 
there are 13 units; five for Spring and Fall semesters, and three for Summer. So, with the latter 
system, if a faculty member taught six sections, his/her instruction load would be 6/13 = .46. If 
he/she were quite involved with nondirect instruction activities such as course/curriculum 
development, student advising, special committee work, etc., his/her instruction load could ' 
increase to 6.5/13 = .50 or 7/13 = .54. It may be noted that some faculty members had assigned 
instruction load time in the .15 to .20 range because of service work, administration work, grant 
work, or nonemployment (during the summer). All Department faculty assigned to 
administrative duties (Head, Program Coordinators, Graduate Coordinator, Teaching Assistant 
Supervisor) have specified assigned administrative loads. For example, a Program Coordinator 
currently has an assigned administration work load of 2/13 = .15. During the past seven years, 
only one faculty member was assigned service work (load of 1/13 = .08). Assigned research 
load time is determined as the supplement of the time for instruction, service, administration, 
and nonemployment. The assigned research load over the past years ranged from .12 to .61 with 
a median of about .36. 

The Self Assessment to be turned in by each faculty member was intended to be a 
summary of his/her strengths and weaknesses that were exhibited during the calendar year under 
consideration. The summary was limited in length to a maximum of four double-spaced typed 
pages. For some faculty members, the Self Assessment statement provided an opportunity to 
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point out some annual accomplishments about which he/she was particularly proud. [A more 
formal self assessment approach is described by Centra (1993, chp. 5).] 



Annual Faculty Summaries 

Two summaries of numerical information pertaining to faculty activities for the year 
under consideration were prepared. One summary pertains to instruction information. The 
column headings are CH (credit hours generated), IL (assigned instruction load, as a percent), 
CH/IL (an index, rounded to the nearer tenth, reflecting instruction productivity), six items from 
our student evaluation form (row entries are the means - to the nearer tenth - of the items 
scored using a 1 to 5 scale). [The six items reflect one general instructor quality, one general 
course quality, and four specific items pertaining to instructor quality. These six items were 
selected by the Department faculty as a whole.] Five-point descriptions of the 1998 CH/IL index 
values, of the student evaluation item (#36), and the percent of As given in the classes taught 
which collectively reflect general instructor quality are given in Table 1. [The reason that percent 



Insert Table 1 about here 



of As given was considered is to see if an instructor’s method of assessment of student learning 
resulted in some variability of student performance. Also, it was viewed as a positive if an 
instructor gave a relatively low percent of As but received high student ratings. It is recognized 
that more variability in student performance would be expected in introductory graduate-level 

courses tl'^an in advanced graduate-level courses.] 

The second annual summary pertains to research involvement and some other 
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professional activities. The column headings of the tabular summary are assigned research load, 
books (authored, edited), book chapters, journal articles, reviews, computer programs/yideos/ 
tests, (complete) papers presented, workshops conducted, other presentations, and number of 
doctoral students graduated. For books, chapters, and articles it is indicated whether the faculty 
member is sole author, senior author, or junior author. For papers, workshops, and presentations 
it is indicated if they were at the international, national, or regional/state level. 

Faculty Review Team 

Near the end of each calendar year. Department Full Professors vvere asked if they would 
be inclined to serve on the Faculty Review Team. Typically, six professors agreed to serve. 

Half-day meeting times were scheduled for Team members to review information in the faculty 
members’ three files. The files of each faculty member were reviewed by two Team members. 

Each Team member was requested to prepare a written statement that was summative with 
respect to activities in instruction, research, service, and administration (if applicable) for the 
year under consideration. A formative statement regarding future effort was also to be prepared. 

One of the two Team members was responsible for preparing the final summative and formative 
statements. Each Team member was scheduled to review six or seven faculty members. Care 
was taken in making review assignments from year to year in that a Team member’s review of 
faculty members was rotated as much as was reasonable. 

Head Review 

The three files of each faculty member were reviewed by the Department Head with respect to 

instaiction, research, service, and, if applicable, administration. What was looked for in each category 
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is briefly described in Table 2. A five-point rating scale was used: Outstanding (5), 



Insert Table 2 about here 



Excellent, Good, Adequate, and Below Adequate (1). A somewhat iterative process was used in 
that each faculty was given a rating in the three (or four) categories. After those ratings were 
determined for all faculty members, they were reexamined with possible changes in the ratings - 
a sense of “relative fairness” was considered. Then a “composite rating” was determined for 

/ 

each faculty member; again some iterations were typically called for. Considerations made to 
arrive at the composite rating included assigned time, professional citizenship, grant 
involvement, and professional development, as well as the summative and formative statements 
of the Review Team. Below are five examples of 1998 calendar-year assigned ratings: 



Tn^tnic 


Res 


Serv 


Admin 


Overall 


3+ 


4 


4 




3+ 


4+ 


2 


4 


4 


4 


2 


4 


2 




3 


3 


4+ 


5 


4+ 


4 


3+ 


3 


3+ 




3+ 



A summary of the 19 1998 Department faculty ratings is given in Table 3. Each faculty member 



Insert Table 3 about here 



received the Head’s critique with the indicated ratings along with the Review Team summative 
and formative statements. 
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Additional Faculty Information 

It was felt that determination of faculty raise monies should be based on some 
information in addition to that for the calendar year under consideration. Two types of 
information were considered. One type is a faculty member’s work over the past three (or five) 
years. With regard to instruction, four numbers were determined: (a) mean CH/IL, (b) mean 
(student evaluation) Item 36 score, (c) mean percent of As given, and (d) number of doctoral 
graduates. A summary of these numbers for 19 Department faculty members is given in Table 4 
for three calendar years, 1996-98. 



Insert Table 4 about here 



With regard to research, three (and five) year summaries of research involvement and 
some other professional activities (as mentioned in the earlier section. Annual Faculty 
Summaries) were completed. From these cumulative summaries, another summary was 
developed with a focus on a writing-productivity index, WP/RL. This index is based on a 
developed point system that is given in Table 5. A WP/RL index value is calculated by 



Insert Table 5 about here 



dividing the total number of points accumulated over the three (or five) years by the mean 
assigned research load (in percent); the index value is rounded to the nearer tenth. The five- 
point summary of WP/RL values for 16 faculty members for 1996-98 is as follows: 

ERIC 
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Max 


1.6 


^75 


1.3 


^50 


0.8 


C25 


0.7 


Min 


0.3 



where “C” denotes centile. [For additional quantifications of research productivity see Braxton 
and Bayer (1986).] 

The other type of information used in addition to faculty performance during the calendar 

/ 

year under consideration relates to salary compression. For 1998 the 1997-98 academic year 
salaries for 203 College of Education faculty were available by number of years in rank. Each 
Department faculty member was identified within a group of 15-20 COE faculty - usually this 
meant considering all other COE faculty with the same number of years in rank plus those . 
faculty with one less and one more year in rank. The centile rank of each Department faculty 
member was thus determined relative to 15-20 COE faculty. The centile ranks for 19 1998 
Department faculty members are summarized in Table 6. 

Insert Table 6 about here 



Allocation of Raise Monies 

The amount of an academic year raise for a given faculty member may depend upon four 
things; (a) activities done during the year under consideration; (b) activities during the previous 
three (or five) years; (c) extra contributions, awards, and/or recognitions; and (d) salary 
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compression. These will be discussed in turn. The dollar allocation for the year’s activities was 
done in a stepwise manner. First, each faculty member with a composite rating of 2+ or higher 
was allotted the maximum of a percent of the previous year salary, X, and a dollar figure. This 
initial percent and initial dollar figure depended upon the total number of raise dollars allocated 
to the Department. For example, for a recent year when the total raise percent for the 
Department was 5.2, the initial percent was 2.5 and the initial dollar figure was $1000. For the 
most recent year the total raise percent was 4.0; the initial dollar figure was $500 (no initial 
percent was considered). For composite ratings higher than 2+, percents of X were added for 
each higher rating. Staying with 1998, for faculty with a composite rating of 3 or higher .02o'x 
was added, for 3+ or higher .005X was added, and for 4 or higher .006X was added. The 
percents of X used in a given year depended upon the number of raise dollars available. Various 
percents were tried in a given year; the percents used for 1997 were .025 (for a composite rating 
of 2+ or higher), .015 (> 3), .010 (> 3+), .010 (> 4), and .010 (> 4+). Iterations were needed to 
arrive at the percents. For 1998 the final total of the three sums ($500 + .020X, $500 + .020X + 
.005X, and $500 + .020X + .005X + .006X) amounted to approximately 3.2% for the 
Department faculty. The remaining .8% was distributed according to the three remaining 
considerations. 

As we all know, even though faculty members may very well work at a steady pace over 
a few years, productivity may be low (or high) for a given year. Thus, it appears reasonable to 
consider activities over the previous, say, three years. This is the second thing taken into 
consideration in allocating raise monies. Two summaries that cover the three previous years 
were examined. One pertains to instruction information: mean CH/IL, mean on (student 
evaluation) Item 36, mean percent of As, and number of advised doctoral graduates -- refer back 
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to Table 4. The second summary pertains to the writing-productivity index, WP/RL. 

The third consideration in raise allocation involves the identification of a faculty 
member’s “extra” contributions - during the calendar year under consideration or during the 
past three years. Such contributions include activities that may be hidden in the reporting of 
instruction, research, service, and administration work. Examples of such are: special grants, 
important committee work, special awards/recognitions (at local, regional, state, national, or 
international levels), department nurturing, program/department leadership, etc. The second and 
third considerations led to allocations of dollar amounts (in “units” of $300 for 1998) to the 

/ 

judged “deserving” faculty members. 

The fourth consideration in raise allocation is salary compression. As discussed at the 
end of the immediately preceding section, faculty salary centile ranks are determined (relative to 
COE faculty academic year salaries with the same (±1) years in rank). Those faculty who made 
reasonable professional contributions over the most recent three years but who were at the 49th 
centile or lower were allocated “extra” raise amounts; greatest allocations were given to those 
faculty members whose salaries were at the 19th centile or less (assuming, of course, that their 
professional contributions were judged to be reasonable). Allocations for salary compression for 

1998 ranged from $200 to $500. [A recently proposed approach to determining salary 

compression was advanced by Toutkoushian (1998).] 

Table 7 is provided to give the reader an idea of how salary raises varied across the 19 

Department faculty members for 1998. 



Insert Table 7 about here 
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Concluding Comments 

Faculty evaluation is a very subjective process. Even though faculty products may be 
counted, student evaluation item means may be calculated, productivity numerical indexes may 
be considered, and various other quantities may be counted and recorded, subjective judgment 
on the part of the evaluator plays a big role in the process. In order for the judgments to be 
reasonable and fair, the evaluator needs to have access to as much information on each faculty 
member as is reasonable. Obtaining information, that is making assessments, is a necessary first 
step in the evaluation process. As suggested earlier, it is reasoned that assessment information 
should be collected not only for the calendar year under consideration, but for at least two years 
prior as well. 

A desirable aspect of faculty evaluation is to have more than one individual (such as the 
department head) involved in the faculty evaluation process. The use of a faculty committee to 
serv'e as a review team gives the evaluation process more credibility. And the added perspective 
may very well highlight some aspects of faculty performance/productivity that may be 
overlooked by the department head. 

As a department head and final evaluation decision maker, I personally have taken, in 
most senses, the view of what I would like to have done for me as an evaluatee. Part of this 
perspective is the provision of evaluation information available to each faculty member. Each 
year that I served as Department Head, I made all numerical information available to each and 
every faculty member. The information made available includes numerical Department 
summaries as well as numerical information on individual faculty members - only the name of 
the faculty member requesting the information is indicated on the summary of numerical 
information for all faculty members. Much to my surprise and chagrin, very few (less than 20%) 

ERIC i c: 



13 



er|c 



of the faculty members expressed an annual interest in seeing the information. 

It is recognized that the two proposed numerical indexes (CH/IL and WP/RL) are not 
without deficiencies. There are some aspects of instruction that are not considered in the 
determination of a CH/IL value; for example, level of course, type (e.g., seminar, lecture) of 
course, demand on instructor, and course audience (department/program majors versus students 
being “served”). Also, the WP/RL index does not consider quality of publications, length of 
publications, citations of publications (Lindsey, 1989), publisher, meeting status for presented 
papers, etc. 

Anyone who has been “around” a university knows that there is some variability in 
performance and productivity across faculty members in most departments. Variability is 
expected to exist with each of the instruction, research, service, and administration categories as 
well as across the categories. [Whereas, the range of the composite ratings was only 3 to 4 for 
1998, the range for 1997 and 1996 was 1 to 4-^ - 1998 was an exception over the past seven 
years.] Different faculty contribute in different ways. Thus, it appears desirable to have an 
evaluation process that, in general, will reveal such variability. 

It is also recognized that there was different emphases on different aspects of faculty 
productivity in different departments. Different cultures exist in different departments. Whereas 
in my department there is considerable variability in what reflects noteworthy contributions, 
especially in the research category, avenues of publication (e.g., limited relevant journals) may 
be much more restricted for other departments. I would maintain, however, that such restriction 
could be “fit in” with the approach described herein. 

It is expected that not all faculty members being assessed and evaluated will be satisfied 
with their evaluation (Ormrod, 1986). In my experience, I have heard concerns about the 

16 
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overemphasis on research, the overemphasis on instruction, and any emphasis at all on service 
(because virtually no one is budgeted for service work). As many administrators have 
experienced, you can’t please them all! 

In sum, evaluation of university faculty for the purpose of determining annual raises is 
complicated, involved, and sometimes perplexing. The evaluation process is, indeed, a tough 
task. 
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Table 1 

ELve-Pflint Descri ptions for 19 1998 Faculty Memh^r^ 





CH/IL 


Item 36 


% As 


Max 


20.4 


4.8 


100 


^75 


10.5 


4.5 


90 


C50 


9.2 


4.2 


78 


^25 


7.0 


3.6 


62 


Min 


3.5 


2.9 


33 



Note: “G” denotes centile. 
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Table 3 



Frequencies of 1998 Faculty Ratings 



Rating 



5 

4+ 

4 

3+ 

3 

2 + 

2 



Instruc Res. 

4 2 

5 8 

2 1 

7 2 

1 5 



■Sery Admin 

1 

2 2 

4 6 

2 

9 1 

1 



Overall 



10 

6 

3 



19 



18 



19 



9 



19 




22 



Table 4 

Summary of 1996-98 Instructioa In formatiQ i i ( N = 1 91 





Mean 

CH/IL 


Mean 
Item 36 


Mean 
% As 


No. Doc. 
Graduates 


Max 


18.3 


4.6 


100 


11 


C75 


10.5 


4.4 


88 


3 


^50 


9.2 


4.1 


76 


2 


^25 


7.6 


3.8 


63 


1 


Min 


3.5 


3.0 


48 


1 



Note: “C” denotes centile. Only 10 faculty members served as doctoral student advisors. 





Table 5 



Points Allotted for Writings 



Books Authored (1st edition) 
Sole 
Sr 
Jr 

Books Authored (later editions) 
Sole 
Sr 
Jr 

Textbook Manuals/Tests 
Sole 
Sr 
Jr 

Books Edited 
Sole 
Sr 
Jr 

Chapters 

Sole 

Sr 

Jr 

Articles (refereed) 

Sole 

Sr 

Jr . 

Papers (complete) 

Others Writings 
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30 

20 

15 

8 

6 

4 

8 

6 

4 

8 

6 

4 

8 

6 

4 

6 

4 

2 

2 

1 



Table 6 



Mi.mhers of Centile Ranks of 19 1998 Department Fa cvdtv Members 



Centile Rank 
90-99 
80-89 
70-79 
60-69 
50-59 
40-49 
30-39 
20-29 
10-19 
0-9 



Number 

2 

1 

2 

4 

5 
1 
1 
1 
2 
0 
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Table 7 



J998 Raise Allocations for 19 Faculty Members 





_% 


No. Pollans 


Max 


5.5 


4300 




4.3 


2700 


o 

O 


4.1 


2300 


C25 


3.5 


1800 


Min 


1.0 


500 



Note: “C” denotes centile. 
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